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Resolution 

ESTABLISHMENT OF JOINT COORDINATING COHMITTEE 
FOR OCCUPATIONAL EDUCATION PROGRAMS 
MARYLAND STATE BOARD OF EDUCATION - 
MARYLAND STATE BOARD FOR COMMUNITY COLLEGES 



The Maryland State Board for Community Colleges and the Mary- 
land State Board of Education, in order to improve, expand, and 
coordinate their individual and joint efforts in the develop- 
ment, achievement, and support of high-quality occupational 
programs and services in postsecondary education for the 
citizens of Maryland; and 

A joint commitment of these two agencies to regular and syste- 
matic cooperation and coordination will assure the achievement 
of certain key objectives; 

NOW, THEREFORE, BE IT RESOLVED That the State Board for Conmunity Colleges 
and the Maryland State Board of Education will link efforts to 
achieve the following objectives: 

1. Coordinate staff efforts through active dialogue and 
i nf ormat i on - shari ng concern i ng curri cul urn, f aci 1 i t i es , 
personnel development, fiscal matters, and appropriate 
technical assistance in the occupational area; 

2. Coordinate appropriate interrelated informational compo- 
nents within information systems; 

3. Systematic sharing of relevant communicction pertaining to 
specific occupational projects undertaken by community 
colleges; 

4- Jointly develop, publish, and disseminate informational 
program materials and brochures highlighting occupational 
programs in Maryland community colleges; 

5. Establish a joint coordinating committee for occupational 
education to foster interagency planning on annual and 
lonq-range master planning. 



Approved: 

State Department of Education, April 28, 1976 
State Board for Community Colleges, May 13, 1976 
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PROJECT SUMMARY 
FISCAL YEAR 1987 



PROJECT TITLE 

Management Information System and Program Evaluation, Activity 
No. 3.IM.5.2. 

PROBLEM 

A valid and reliable student data base Is needed for planning, decision 
making, and Vocational Education Data System (VEDS) reporting. A systematic 
and objective procedure Is needed to evaluate the quality of occupational 
programs In Maryland community colleges. 

OBJECTIVE NO. 1 

To conduct the following statewide projects, Including the preparation 
of computer files and publication of a report for each study: 

Credit enrollment 
Degree recipients 
Graduate follow-up 
Leaver (entrant) follow-up 
Employer follow-up 
Discipline cost analysis 
Continuing education data system 
Program inventory 

PROCEDURES AND RESULTS 

Credit Enrollment . Opening Fall enrollment is reported using a system 
of computer tapes from the colleges. In addition to demographic data about 
each student, the student's program of study and credit hours carried are 
reported. All seventeen community colleges submitted tapes or coding forms 
in Fall 1986. Data were published in the State Board for Community Colleges 
(SBCC) State Plan, the Databook, the Program Data Monitoring System, and a 
Technical Memo concerning enrollment trends. 

Certificates and Degrees Awarded . The number of students who complete 
programs during the fiscal year are reported by college by program. Trends 
in the mix of occupational and transfer degrees are analyzed. Colleges 
submit the degree data on computer tapes or coding forms with one record per 
graduate. Degree trends were analyzed in the Databook and in a Technical 
Memo shared with the colleges. 

Graduate Follow-Up . The statewide graduate follow-up study surveys 
pev'sons about nine months after program completion. The study is a joint 
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project of the community colleges and the SBCC and uses a standard question- 
naire mailed in odd-numbered years. In FY 1987, 7,530 graduates from 1986 
were surveyed. An adjusted response rate of 55 percent was achieved- Each 
college has received a printout of its results, the results from graduates 
of similar peer colleges, and the statewide results. The n^ data have been 
added to the Program Data Monitoring System and a summary report of the 
survey will be sent to the DVTE. 

Entrant Follow-Up , Follow-up surveys of all entering students are 
conducted two to four years after entry into the community colleges. All 
first -time students from Fall 1982 were surveyed in Spring 1984 and results 
have been used in a number of studies examining why students attend com- 
munity colleges, why they leave before graduation, and how long they take to 
giraduate. During FY 1987, the SBCC and the Maryland Community College 
Research Group revised the questionnaires and procedures for a survey of all 
entering 1984 students who will be surveyed four years later in Spring 
1988. 

Employer Fol low-Up . Surveys of the employers of community college 
graduates have been conducted every other year in conjunction with the 
surveys of graduates. The study surveys all employers of graduates from 
occupational programs who were employed full-time and working in a field 
related to their college program and who gave the college permission to 
contact their employer. During FY 1987, the employer survey was revised by 
the SBCC and the colleges, and employers of the 1986 graduates were sur- 
veyed. A report utilizing the data from this employer survey will be sent 
to the DVTE in FY 1988. 

Di scipline Cost Analysis . Costs and full-time equivalent students are 
reported for each discipline at each college. The data enable an analysis 
of trends in costs per FTE student in occupational disciplines. Colleges 
are provided with trend reports for their college, similar size colleges, 
and statewide. A Discipline Cost Analysis Manual has been developed and is 
revised periodically to clarify the procedures for preparing the cost data. 
Colleges report the daU to the SBCC on a standard format tape in order to 
facilitate processing and accuracy of the data. 

Continuing Education Data System . Enrollments and contact hours are 
reported* to the SBCC for each approved noncredit course. Reports are gen- 
erated showing the number of registrants and full-time equivalent reports 
are sent to the colleges and to the DVTE at the end of each year. Summary 
reports of enrollments in noncredit courses are included in the annual SBCC 
Databook. 

The State Board for Community Colleges has implemented a computerized 
system for colleges to submit enrollment, course, and student demographic 
data in continuing education. This tape system provides software that 
generates a standard format file and producej management and aid transmittal 
reports at each college. 



Program Inventory , The SBCC Program Inventory was updated in FY 1987 
to reflect programs that were ^dded, discontinued, or made inactive. The 
Program Inventory is the backbone of the SBCC Information System and is 
critical to the Program Data Monitoring System and to student follow-up 
studies. A crosswalk is used for the new federal Classification of Instruc- 
tional Programs (CIP). Colleges will continue to report their enrollment, 
degrees, discipline cost analysis, employee data, and follow-up surveys 
using the present six-digit State Board for Community Colleges and State 
Board for Higher Education (SBHE) subject codes- The SBCC and SBHE trans- 
late these program codes to CIP codes for any federal report which requires 
CIP CO s. 

Common Data Elements . In a statewide system of locally governed com- 
munity colleges, each institution is free to develop its own data processing 
system. In order to ensure that data from eacn institution are comparable 
and to promote more efficient development of data systems, the Association 
of Data Processing Directors of Maryland Community Colleges has produced the 
Course Master Data Elements and Student Data Element Dictionary. This Data 
Element Dictionary has been continuously updated by the data processing 
directors and maintained by the State Board for Community Colleges on the 
SBCC computer file. 

OBJECTIVES NO. 2 AND NO. 3 

To run the SBCC Program Data Monitoring System and review each of the 
career programs by June 15, 1987. (See Appendix A for a description of the 
SBCC PDM System.) 

To publif>h an in-depth evaluation of 50 community college programs by 
March 15, 198/ and a statewide evaluation of paralegal programs. (See 
Appendix B for a description of the SBCC Program Evaluation System.) 

PROCEDURES AND RESULTS 

Program Data Monitoring System . The 1987 printout of the Program Data 
Monitoring System was sent to college occupational deans and institutional 
researchers for confirmation of the data. Upon receipt of the corrections, 
a final printout was made and a copy was sent to the DVTE with one page per 
program at each college. 

Enhancements are made to the PDM System each year. This year data were 
added about Fall 1986 enrollment, FY 1986 awards, and FY 1986 discipline 
costs. Also, changes were made so that the discipline cost data are now 
calculated based on six-digit HEGIS numbers, not the four-digit numbers as 
in the past. The number of credit hours generated in a discipline is also a 
new addition to the PDM display. This is shown for each discipline, both at 
the college and statewide. 

Program Evaluation . The system for program evaluation is described in 
length in the SBCC Instructional Progran Manual. Briefly, the SBCC reviews 
each program at each college every April using the Program Data Monitoring 
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System. The PDM System displays trends in enrollments, awards, employment 
in field of training, student satisfaction, discipline costs, and job open- 
ings. Programs that appear to need a qualitative evaluation are identified 
and a letter asking specific questions is sent to the college presidents. 
The colleges prepare written responses, and the responses are compiled into 
a Program Evaluation Report reviewed by the SBCC in January. 

The 161 -page 1987 Prociram Evaluation Report was published in April and 
sent to the DVTE. After the ninth full cycle of program e/aluations, 431 
programs have been evaluated. Many programs have been improved through 
changes in curriculum, staffing, or recruitment. In addition, 84 programs 
have been discontinued. Table 1 shows the number of programs evaluated by 
year and by occupational program area. 

In June 1987, 44 programs were identific^d for a qualitative evaluation 
in the tenth cycle of the Program Evaluation System to be conducted in 
FY 1988. Table 2 gives the names of the programs identified for a college 
evaluation. 



Tib1« 1 

PROGRAM AREAS DESIGNATED FOR EVALUATION 
HARYLAND COMMUNITY COLLEGES 
1978 • 1986 



Nu«b«r 



Progruu 





Programs 
1 i 


1978 

nr 


1979 

nr 


1989 
"T 


1961 




1983 


IT 


IddS 
~r 


idd6 

~r 


Total 
i % 


« % 


TRANSFER 


87 


21 


0 


1 


16 


3 


14 
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11 


7 


6 


66 


15 


0 


0 


OCCUPATIONAL 
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10 


10 
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15 


12 


10 


13 
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15 
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25 


16 


19 


Data Processing 


2S 
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1 


1 


0 


0 


1 


17 


C 


27 


6 


1 
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Health Services 


68 


17 


a 


6 


1 


11 


7 


6 


6 


9 


11 


65 


15 


17 


20 


Mechanical & Engineering 


62 


15 


11 


14 


8 


9 


9 


7 


13 


5 


10 


85 


20 


21 


25 


Natural Science 


15 


4 


2 


4 


3 


3 


2 


2 


0 


3 


2 


21 


5 


10 


12 


Public Service 


49 


-1? 


_4 


_5 


7 


10 


_4 


J5 






_$ 


Jl 


J4 


Ji 


23 


TOTAL 


409 


100 


37 


45 


48 


52 


48 


48 


50 


53 


50 


431 


100 


84 


100 
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Table 2 



PROGRAMS DESIGNATED FOR QUALITATIVE EVALUATION 
MARYLAND COMMUNITY COLLEGES 
1987 



CHESAPEAKE 
Radiologic Technology 
Develop- .ntal Disabilities 



/ . iGANY 

nursing, Practical 5209-01 

Automotive Technology (Statewide) 5306-01 

ANNE ARUNDEL 

Human Services ' 5216-01 

Designing and Drafting Techffology 5303-01 

BALTIMORE 

Banking and Finance 5003 01 

Nursing, Registered 5208-01 

Medical Records Technology 5213-01 

Construction Technology 5317-01 

Heating, Air Conditioning, and Refrigeration 5317-02 

CATONSVILLE 

Television Technology (Statewide) 5008-02 

Nursing, Registered 5208 01 

Interpreting for the Deaf 5599-05 

CECIL 

Nursing, Registered 5208-01 

Sanitary Technology 5408-01 

CHARLES 

Office Technology 5005-01 

Nursing, Registered 5208-01 



5207-01 
5503-02 

DUNDALK 

Business Administration Transfer Area 4970-01 

Physical Fitness Technology (Statewide) 5299-11 

Heating, Ventilation and Air Conditioning 5317-02 



ErJc 12 



ESSEX 

Veterinarv Technology (Statewide) 5206-01 

Radiog aphy 5207-01 

Nursing, Registered 5208-01 

FREDERICK 

Nursing, Registered 5208-01 
GARRETT 

Maintenance Technology 5312-02 

Human Services 5503-01 

HAGERSTOWN 

Radiologic Technology 5207-01 

Nursing, Registered 5208-01 

HARFORD 

General Studies 4950-01 

Photography 5007-01 

Computer Operator 5104-01 

Nursing, Practical 5209-01 

HOWARD 

Vision Care Technology (Statewide) 52i2-01 

Biomedical Engineering Technology (Statewide) 5399-02 

Plant Science 5402-01 

MONTGOMERY 

Nursing, Register:d 5208-01 

Medical Assistant (Statewide) 5214-01 

Corrections 5505-02 

PRINCE GEORGE'S 

Engineering Transfer Area 4940-01 

Nursing, Registered 5c08-01 

Medical Records Technology (Statewide) 5213-01 

WOR-WIC TECH 

Hotel -Motel -Restaurant Management (Statewide) 5010-01 

Nursing, Registered (Statewide) 5208-01 
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Appendix A 



A GUIDE FOR USERS 
OF THE PROGRAM DATA MONITORING SYSTEM 



The State Board for Community Colleges Program Evaluation System was 
developed in cooperation with the community college presidents and approved 
by the Board in 1978. The system involves two steps: quantitative evalua- 
tion of each program each year by the SBCC, and qualitative evaluation of 
selected programs each year by the community colleges. The following infor- 
mation is given to assist in the interpretation of data on the Program Data 
Monitoring (PDM) printout (see page 9): a display of enrollment, awards, 
follow-up, discipline cost, and manpower information. 

Program Number and College Program Title , These items are taken from the 
current SBCC Program Inventory, Data are only shown on the PDM printout as 
related to programs and not according to program options. Data related to 
program options are included as part of the overall program. For example, a 
college may have a program in Electronics Technology with an option in Digi- 
tal Electronics, Data i elated to the Digital Electronics option are in- 
cluded in the Electronics Technology data. 

Enrollment and Awards , Enrollment and awards data are obtained from Enroll- 
ment Information System (EIS) data tapes and from the Degree Data System 
tapes, respectively. The "FT ENR-TO-AWARD RATIO" is derived by taking 
one-half of the full-time enrollment in a given year and dividing that 
figure by the number of graduates two years later. For example, a program 
that had 100 full-time students and 25 graduates two years later would have 
a ratio of 2:1, All student and discipline cost information on the PDM 
printout is submitted by the colleges, and changes are not made in the data 
without consulting the colleges. 

Similar Programs , Enrollment and award data from the most recent year are 
shown for each college having the same program in order to show the scope of 
the programs at adjacent colleges and statewide. 

S tudent Follow-Up Studies , Student follow-up studies are conducted jointly 
by the college and the SBCC, with the college responsible for mailing the 
questionnaires to students. Graduates are typically mailed surveys nine 
months after graduating. All percentages in the follow-up section are 
simple percentages of the number of respondents, except the data "Among 
Full-time Employees" where blanks have been excluded from the ^nominator to 
arrive at the percentage. Graduate follow-up data are generally not inter- 
preted when there are less than two respondents. 

Annual Job Openings to 199G . The data were developed by the Maryland De- 
partment of Employment and Training, Research and Analysis Division, The 
projections include an'^ual openings due to growth and labor force separa- 
tions by occupation through 1990, Statewide data are shown for all programs 
where relevant occupational data are available. In addition, Baltimore area 
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data are shown for the Community College of Baltimore, Catonsville Community 
College, Dundalk Community College, and Essex Community College. The Balti- 
more area data include Baltimore City, plus Anne Arundel, Baltimore, Csr- 
roll, Harford, and Howard Counties. Regional data are provided for Charles 
County Community College (Calvert, Charles, and St. Mary's Counties), Chesa- 
peake College (Caroline, Dorchester, Kent, Queen Anne's, and Talbot Coun- 
ties), and Hor-Wic Tech Community College (Somerset, Wicomico, and Worcester 
Counties). For all other colleges, county data are provided. 

Discipline Credit Hour Cost . Data are shown as reported in the SBCC Disci- 
pline Cost Analysis for the fiscal years shown. Basically, the discipline 
cost analysis accounts for all expenditures at the colleges (including 
federal) and attributes them to direct costs, additional direct costs, and 
indirect costs. Direct costs are a function of faculty compensation and 
class size. Additional direct costs include supplies and materials related 
to the teaching of that discipline, such as laboratory supplies. Indirect 
costs include general administration costs, including the library, student 
services, and utilities. Indirect costs are applied to all courses in 
proportion to the number of full-time equivalent students in the course. 
Cost data are shown for each discipline at a college and compared to the 
cost of that discipline statewide; costs also ar^ shown for all disciplines 
at a college and compared to the cost of all disciplines statewide. The 
number of student credit hours generated in each discipline, both at the 
college and statewide, also are shown.. 

Procedure for Program Evaluation . Every April the updated PDM printout is 
distributed to academic deans, occupational deans, and institutional re- 
search directors to verify the data. The SBCC staff then identifies several 
programs at each college appearing to be in some difficulty and in need of a 
qualitative evaluation. In general, selection is based upon enrollment, 
awards, and job placement, in that order. Discipline cost information is 
used in the context of the overall discipline cost at the college and the 
average cost of the discipline at other colleges. 

The selection of programs for a qualitative evaluation is assisted by 
"flags" ,shown in the lower right corner of the PDM printout. The flags 
represent criteria checks on certain data, such as "Enrollment decreased at 
least 20 percent and by ten students last year." The flags were set with 
the help of community college academic deans, occupational deans, and insti- 
tutional research directors. The flags are used by the SBCC staff to assist 
in selecting programs for a qualitative evaluation but are not the sole 
criteria for requesting a qualitative evaluation by the college. 

The SBCC staff constructs questions to be addressed in the qualitative 
evaluation. Upon approval by the SBCC in June, the questions are sent to 
each college president. Colleges conduct the qualitative evaluations, 
respond to the questions raised, and submit a report to the SBCC. The SBCC 
publishes a report of all questions and responses. The report is distrib- 
uted to the community colleges, the State Board for Higher Education, the 
State Department of Education, Division of Vocational -Technical Education, 
and the Maryland State Council for Vocational -Technical Education. 
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CHARLI 

nn? 



ES 

ER I CK 



HACKRSTOUN 
HARFORD 
HOWARD 

noNTConERv 

PRINCf CEORCE' 
UOR-uTc TECH 



120 
S2 
4ft 
SI 

il 

17 
ft7 
47 
SO 
I4t 
17 



SI 

35S 
13S 
4TS 
7^ 
IS 
204 

2f 

ITS 

"I 
413 
30 



20t 
410 
101 

zil 

114 
ft4 
24ft 

Hi 



72 
ftS 

31 

!i 
» 

OS 

72 
74 
I Of 
27 



TOTAL ALL COLLEGES 


S34 


24S3 


3317 




72f 


DjSCI^INC CREDIT HOUR 


DATA 


NURSING. 


RN 








FY63 


FY04 


FYSS 


FYOft 


S3-Sft < 


HIGHER ED FRICE INDEX 


*240 


*2S3 


'27S 


'isi 


"**1SX 


OTATEUirC COOT- 
ALL oiociplim:s 


«I4 


SI01 


kISt 


SII7 


♦24X 


THIS DISC!«*LINE 


0152 


SIftO 


S'7ft 


Sift 


♦31X 


COLLEGE COST 

ALL DISCIPLINES 


SlOft 


S112 


S123 


SI3t 


♦31X 


TMIO DISCIPLINE 


SI4ft 


SI30 


Siftt 


S204 


♦ 40X 


STUDENT CREDIT HOURS> 
STATEWIDE 42 


,44t 


44. SOS 


40.102 


33.242 


•22X 


COLLECE 3«34t 


3.3S0 


2.t2l 


2.213 


-34X 


THESE COSTS ARE DISCIPLINE COSTS. NOT PROCRAn COSTS. 
A SINGLE DISCIPLINE nAY SUPPORT SEVERAL P'OCRAnS 





i? 



•if 



72f 



POn QUANTITATIVE CRITERIA CHECK4 INDICATE* 

ENROLLnENT DECREASED AT LEAST 10X AND SY IS STUDENTS LAST YEAR 
DISCIPLINE COST IS AT LEAST 40X OVER COLLEGE AVERAGE COST 
DISCIPLINE COST INCREASED AT LEAST 20X LAST YEAR 



PROCRAn EVALUATED UITH SSCC PROCRAn (VALUATION OYSTEn OS 
ft Or^cTpLINE~COST*AN^LySIS^~OATA^ARE~$ Sy'cOLLIGeT 
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Appendix B 



USING INFORMATION 
TO SUPPORT INSTRUCTIONAL PROGRAM EVALUATION 



Program evaluation In Maryland community colleges Is part of an Inte- 
grated system for program Implementation and evaluation, with both pieces 
woven together by an extensive data base. The process for Implementing and 
evaluating Instructional programs In Maryland community colleges Is diagram- 
med on the following page. New programs begin with local needs. Faculty 
members and administrators see educational needs among students and employ- 
ers and respond with program Ideas. By December, the college submits the 
titles for Its proposed programs to the State Board for Community Colleges 
(SBCC) and the titles are printed In the State Plan for CoMMinity Colleges 
In Maryland. In June, the colleges submit Letters of Intent that describe 
the goals and nature of the proposed programs. The SBCC acts on the Letters 
of Intent at Its July meeting, raising questions as necessary and making 
suggestions for program development. Often the questions arise from the 
Program Data Monitoring System. For example, a college could propose a new 
program In Recreation Technology; statewide data about employment of gradu- 
ates of existing Recreation programs may Indicate that graduates of the new 
program would have great difficulty getting jobs. 

In November, the colleges submit full program p: , sals. The proposals 
are analyzed by the SBCC staff with an eye toward employment possibili- 
ties, student Interest, and cost. Ayaln, the Program Data Monitoring System 
Is used to establish benchmarks for reasonable projections In the proposals. 
The SBCC acts on the program proposals at the December meeting. Following 
SBCC action, the programs are sent to the State Board for Higher Education 
(SBHE) for approval. The division of Vocational -Technical Education (DVTE) 
of the Maryland State Department of Education also reviews the proposals for 
federal funding. Approved programs generally are Implemented the following 
September, about 15 months after the Letter of Intent was submitted. 

Program evaluation is a two-step process In Maryland community col- 
leges. The first step Is a quantitative evaluation. Each April the SBCC 
staff evaluates each program at each college using the Program Data Monitor- 
ing System. The PDM System produces one computer page for each program and 
displays trends in enrollment, degrees, and discipline costs. Employment 
and transfer information from entrant and graduate follow-up also is shewn. 
In May, the printout is sent to the colleges for ve'^lfication. 

In June, the SBCC requests colleges to conduct a qualitative evalua- 
tion, the second step in the process. Typically, the local qualitative 
evaluation is done with a committee of faculty, academic administrators, and 
institutional research staff. The committee draws on their expediences, 
conducts a deeper analysis of existing data, and seeks more Information from 
students or employers. The qualitative evaluations are reviewed and com- 
piled by the SBCC and distributed to the SBHE and the DVTE. 



- 11 - 



ERLC 



IV 



INSTi^UCTIONAL PROGRAM IMPLEMENTATION AND EVALUATION PROCEDURE 
MARYLAND COMMUNITY COLLEGES 



Local Needs 


^ 


Program 
Title in 
State Plan 


5 


Letter 
of 




Proposal 
tj 




Proposal 
to 




Program 
Implemented 








Intent 


^ 


SBCC 


> 


SBHE and DVTE 


=> 

























DATA: 
Enrollment 

Degrees and Certificates 
Entranv Fol low-Up 
Graduate Fol low-Up 
Discipline Costs 
Manpower (job openings) 



Program Data 
Monitoring System: 

Qualitative Evaluation 

of Each Program 



Qualitative 
Evaluation 
by College 





Report 




Report 




to 




to 


— ^ 


SBCC 


^ 


SBHE and DVTE 



SBCC - State Board for Community Colleges 
SBHE - State Board for Higher Education 
DVTE - Division of Vocational-Technical Education, 
State Department of Education 



Possible 

Program Actions: 
Modify 

Make Inactive 
Discontinue 



